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Abstract 

As part of the Advanced SenSor and Imaging System Technology (ASSIST) program, imaging systems and interface 
requirements are being evaluated in order to enhance a pilot's view of the outside environment under restricted 
visibility conditions. A sequence of simulated images as seen from an aircraft as it approaches a runway was 
obtained from a model of a passive millimeter wave sensor operating at 94 GHz. Atmospheric attenuation is known 
to be a local minimum at this frequency. These images were analyzed to delineate objects of interest such as 
runways, taxiways, buildings, etc. Information about the position and motion of various objects on the ground may 
be computed by tracking the regions in such segmented image sequence. This information can be combined with 
data from other sensors to alert the pilots of any objects which are likely to be a hazard for continuing the flight. 


1. Introduction 


Enhancing the safety of operation of aircraft under restricted visibility conditions has been an important research 
topic for many years (1], There is a great deal of interest in imaging sensors with the capability to see through fog 
and produce a real-world display which, when combined with symbolic or pictorial guidance information, could 
provide the basis for a landing system with lower visual minimum capability than those presently being used 
operationally [2J. An example of the interest in sensor and imaging systems technology is the FAA/DOD /Industry 
program entitled, Synthetic Vision [3]. 


Since the attenuation of radiation in the visible spectrum due to fog is very large [4] (Figure 1), sensors are being 
designed to operate at lower frequencies at which the attenuation is smaller (providing the ability to see through 
fog). One such frequency of choice is 94 GHz since the clear-air attenuation at this frequency is a local minimum 
and fog attenuation is low (Figure 1). In a joint research project, NASA Langley Research Center and TRW are 
developing an imaging system to capture images using a passive sensor operating at this frequency. The goal of 
this project is to establish a foundation for imaging sensor and pictorial graphics display technology and their 
integration into future cockpit systems to enable transport operations under restricted- visibility conditions [2]. 
Extensive enhancement and analysis of images acquired using such sensors is essential in order to identify and track 
various objects such as buildings, runways, vehicles, etc. as the aircraft approaches a runway for landing. 
Information extracted from such an analysis is useful to generate warning signals to the pilot of any potential 
hazards. 

A real-time model of the visible output from a 94 GHz sensor, based on a radiometric simulation of the sensor, has 
been developed. A sequence of images as seen from an aircraft as it approaches for landing has been simulated 
using this model. Thirty frames from this sequence of 200 x 200 pixel images (For: H x V) were analyzed to 
identify and track objects in the image using the Cantata image processing package within the visual programming 
environment provided by the Khoros software system [5|. The image analysis operations are described in this 
paper. 


2. Analysis of Simulated Images 

The image analysis operations include noise filtering, edge detection, edge closing, region labeling, and matching 
of corresponding objects in the image sequence. An example of the iconic Cantata workspace used in this analysis 
is shown in Figure 2. Three frames from the simulated image sequence on which this analysis was performed are 
shown in Figure 3. The image analysis operations are described in this section. 
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Noise Filtering: Since a model for the noise phenomena is not yet available, random noise was added to the images 
during simulation. A simple weighted spatial averaging filter (module marked vconvolve in Fig. 2) [6] was then 
used to decrease the effect of this additive noise. Although this is adequate to filter noise in these simulated images, 
a filter designed to match the noise in actual, real-world images will have to be developed later, when such data 
is available. 

Edge Detection: Edge pixels in the images were detected using the Difference Recursive Filter (DRF; vdrfin Fig. 
2) [7]. This filter, which is based on principles similar to those of the well-known Laplacian of Gaussian filter [8], 
has been shown to be an optimal filter for detection of step edges in white noise [7]. The smoothing function used 
by this filter is a symmetrical exponential function. 

Edge Closing: Edge pixels detected by the DRF filter are not guaranteed to form closed object boundaries. Such 
gaps in boundaries are bridged using an edge-closing operator ( vclose in Fig. 2). The edge-closing operator uses 
as its input two images: the output of the DRF filter and a gradient image (eg., output of a Sobel edge detector - 
vdiff in Fig. 2). It follows a path of maximum gradient between the two end points being bridged. The resulting 
closed edges detected in corresponding images of Figure 3 are shown in Figure 4. 

Region Labeling: Connected pixels belonging to the same object in the image are given a unique label. 4- 

connectivity [6] is used in this labeling scheme (pixels which are connected at the top, bottom, left, and right - but 
not diagonally - are said to be four-connected to the pixel at the center). The region labeling operator ( vlabel in 
Fig. 2) uses the output of the edge-closing module; alternatively, it is also possible to label objects by operating 
directly on the noise filtered image using region growing techniques. 

Region Matching: Objects in consecutive frames of the image sequence are compared to identify corresponding 
objects. The matching ( vrmatch in Fig. 2) is done using a probabalistic rule which is based on the size and shape 
of regions. It computes the similarity of objects in successive frames. To illustrate objects which are matched they 
are assigned the same gray level in both frames in the displayed image. Two such frames after matching are shown 
in Figure 5. 


3. Conclusions 

A software system for analyzing simulated image sequences from a passive millimeter wave imaging system was 
described in this paper. The analysis/tracking system consists of the following stages: smoothing using a spatial 
averaging filter for noise reduction; detection of edge pixels using a recursive filter; bridging discontinuities in 
detected edges using an edge-linking operator; labeling objects in each image in the sequence; and comparing objects 
in consecutive frames to locate corresponding objects. 

The passive millimeter wave imaging system will facilitate operation in restricted visibility conditions. Automated 
analysis of sensor-captured images to locate and track moving objects will provide critical information necessary 
for detecting potential runway incursions. Quantifying the amount of apparent motion of static objects in the scene 
from consecutive image frames is also useful as a cross-check of the position and velocity of the approaching 
aircraft. The preliminary results presented here using simulated images clearly demonstrate the potential of image 
analysis methods for detecting and tracking objects in a dynamic scene. Analysis of real images is well known to 
be a much more difficult task, however, particularly in real-time. To realize a practical system, new vision 
algorithms for analyzing sensor-captured images, and new display concepts for fusing information from various 
sensors, are necessary. 
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Figure 1. Atmospheric effects on electromagnetic radiation [4]. 



Figure 2. Block diagram of the image analysis and tracking operations presented as a Cantata workspace 
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Figure 3. Frames 1, 16, and 30 in the sequence of simulated images. 



Figure 4. Runway, taxiway, horizon, and buildings delineated in the images of Figure 3. 



Figure 5. Two consecutive frames in the image sequence after detection and tracking of objects; 
corresponding objects in the two images are given the same gray level. 
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